Barrier Frank-Wolfe for Marginal Inference
نویسندگان
چکیده
We introduce a globally-convergent algorithm for optimizing the tree-reweighted (TRW) variational objective over the marginal polytope. The algorithm is based on the conditional gradient method (Frank-Wolfe) and moves pseudomarginals within the marginal polytope through repeated maximum a posteriori (MAP) calls. This modular structure enables us to leverage black-box MAP solvers (both exact and approximate) for variational inference, and obtains more accurate results than tree-reweighted algorithms that optimize over the local consistency relaxation. Theoretically, we bound the sub-optimality for the proposed algorithm despite the TRW objective having unbounded gradients at the boundary of the marginal polytope. Empirically, we demonstrate the increased quality of results found by tightening the relaxation over the marginal polytope as well as the spanning tree polytope on synthetic and real-world instances.
منابع مشابه
Marginal Inference in MRFs using Frank-Wolfe
We introduce an algorithm, based on the Frank-Wolfe technique (conditional gradient), for performing marginal inference in undirected graphical models by repeatedly performing MAP inference. It minimizes standard Bethe-style convex variational objectives for inference, leverages known MAP algorithms as black boxes, and offers a principled means to construct sparse approximate marginals for high...
متن کاملA Robust Frank-Wolfe Method for MAP Inference
Finding maximum a posterior (MAP) estimation is common problem in computer vision, such as the inference in Markov random fields. However, it is in general intractable, and one has to resort to approximate solutions, e.g. quadratic programming. In this paper, we propose a robust Frank-Wolfe method [6] to do the MAP inference. Our algorithm optimizes the quadratic programming problem by alternat...
متن کاملLearning and Inference in Latent Variable Graphical Models
OF THE DISSERTATION Learning and Inference in Latent Variable Graphical Models By Wei Ping Doctor of Philosophy in Computer Science University of California, Irvine, 2016 Professor Alexander Ihler, Chair Probabilistic graphical models such as Markov random fields provide a powerful framework and tools for machine learning, especially for structured output learning. Latent variables naturally ex...
متن کاملCorrecting Covariate Shift with the Frank-Wolfe Algorithm
Covariate shift is a fundamental problem for learning in non-stationary environments where the conditional distribution ppy|xq is the same between training and test data while their marginal distributions ptrpxq and ptepxq are different. Although many covariate shift correction techniques remain effective for real world problems, most do not scale well in practice. In this paper, using inspirat...
متن کاملBethe Learning of Graphical Models via MAP Decoding
Many machine learning tasks require fitting probabilistic models over structured objects, such as pixel grids, matchings, and graph edges. Maximum likelihood estimation (MLE) for such domains is challenging due to the intractability of computing partition functions. One can resort to approximate marginal inference in conjunction with gradient descent, but such algorithms require careful tuning....
متن کامل